Decision makers often have to rank by ordinal preference a myriad of options or cases (hereinafter cases). This process is subject to many types of bias and process failures such that the ranking may not be valid or repeatable and a decision maker may be unduly influenced by his or her prior ratings or those of a fellow decision maker. In addition to the lack of outcome optimization, supporters of alternative cases or their constituents may have particular concern with respect to the perception of fairness or procedural justice and may consider that an outcome was capricious, idiosyncratic, unsubstantiated or discordant with objective measures.
Two-dimensional weighting matrices (i.e. of attribute multiplied by weight) have been used both manually and in conjunction with computer technology in decision making processes. However, the two dimensional weighting matrices approach alone does not provide sensitivity analysis, facilitate a process, provide repeatability and validity, nor allow for diagnostic reporting or illustrate discrepancies between inter- and intra-rater performances. Typically decision making that is based on two dimensional weighting matrices has relied on self-assessment by the raters and is subject to manipulation when the weights are known to the raters. Therefore, the two dimensional weighting matrices approach does not address the types of bias and process failures described above.
Another approach that has been used is conjoint analysis. Conjoint analysis is a statistical technique using a multi-attribute compositional model in which respondents are asked to rank or choose between alternatives based on attributes. This approach is limited to a relatively small set of attributes. As the number of attributes to be considered grows, the number of combinations grows very quickly and the information gathering stage becomes complex.
What is required is a decision support tool to improve the reliability, accuracy and repeatability of decisions involving ranking by ordinal preference.